Skill composition is the ability to combine previously learned skills to solve new tasks. As neural networks acquire increasingly complex skills during their pretraining, it is not clear how successfully they can compose them. In this paper, we focus on Multimodal Large Language Models (MLLM), and study their ability to compose skills across modalities. To this end, we design three evaluation tasks which can be solved sequentially composing two modality-dependent skills, and evaluate several open MLLMs under two main settings: i) prompting the model to directly solve the task, and ii) using a two-step cascaded inference approach, which manually enforces the composition of the two skills for a given task. Even with these straightforward compositions, we find that all evaluated MLLMs exhibit a significant cross-modality skill composition gap. To mitigate the aforementioned gap, we explore two alternatives: i) use chain-of-thought prompting to explicitly instruct MLLMs for skill composition and ii) a specific fine-tuning recipe to promote skill composition. Although those strategies improve model performance, they still exhibit significant skill composition gaps, suggesting that more research is needed to improve cross-modal skill composition in MLLMs.

Vision-Language Models (VLMs) have demonstrated impressive performance on zero-shot classification, i.e. classification when provided merely with a list of class names. In this paper, we tackle the case of zero-shot classification in the presence of unlabeled data. We leverage the graph structure of the unlabeled data and introduce ZLaP, a method based on label propagation (LP) that utilizes geodesic distances for classification. We tailor LP to graphs containing both text and image features and further propose an efficient method for performing inductive inference based on a dual solution and a sparsification step. We perform extensive experiments to evaluate the effectiveness of our method on 14 common datasets and show that ZLaP outperforms the latest related works. Code: https://github.com/vladan-stojnic/ZLaP

Sentence Correction using RNN is simple problem in which we provide text data in corrupted form(gramatical mistake,short forms of some words like'ppl' for'people')to the input and output we get is the correct uncorrupted form of that text data.It can be used as a preprocessing step in a language transaltion model where the input language(in corrupted form) can be converted to uncorrupted form and then pass to a model to output the translated text and thus can help in increasing the efficiency of the language translation model.This case study will be useful for increasing the efficiency as many NLP tasks,since any model will learn from uncorrupted correct text and will be able to predict correctly the target task.Moreover it would be useful in text messaging apps where we could enter a corrupted text and it would suggest us the correct uncorrupted text before sending the text to anyone. Since the task at hand comprises of textual data,in which one form of corrupted textual data is to be converted into uncorrupted form while preserving the semantic meaning of the text.The task is similar to a language translation.The task can be converted to DL problem using LSTM's,GRU's and RNN.Since these archtectures help us to take in account the semantic meaning of text and we can use encoder decoder model to encode corrupted text and then decode it to uncorrupted form. We used to different datasets for our task.one The other dataset we got from https://www.cl.cam.ac.uk/research/nl/bea2019st/:fce_v2.1.It contains sentences with spelling and gramatical mistakes.We used subset of 5k datapoints only since we do not want our model to get biased towards one type of dataset.The dataset is in m2 format. Latency: As far as the latecy is concerned.Our model should output quickly(can take seconds) if it is used as preprocessing step for any other NLP tasks.But if it is used in a text messaging apps the output should be quick within milliseconds.